Papers with limited fine-tuning
Entropy-guided Vocabulary Augmentation of Multilingual Language Models for Low-resource Tasks (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual language models (MLLMs) support low-resource languages (LRLs) but LRL words are under-represented in wordpiece/subword vocabularies, leading to low task accuracy . |
| Approach: | They propose an entropy-based vocabulary augmented language model to detect LRL words with undesirable wordpiece segmentations. |
| Outcome: | The proposed model improves performance and limits on wordpiece augmentation strategies for multiple diverse LRLs. |
Less Mature is More Adaptable for Sentence-level Language Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies fine-tune encoders or contrastive learning approaches to learn sentence representations. |
| Approach: | They propose to use sentence-level models to study how sentence representations influence downstream task performance. |
| Outcome: | The proposed models outperform token-level models in terms of time and data efficiency. |